Streaming

BindAI v0.1 provides an HTTP streaming endpoint for agent execution through the REST API. Streaming allows a client to receive agent output progressively instead of waiting for the complete response. The endpoint is:

POST /api/v1/agents/{agent_name}/stream

The endpoint requires API-key authentication and returns a text-based streaming response.

Streaming Architecture

A streaming request follows this general flow:

Client
   |
   | POST /api/v1/agents/{agent_name}/stream
   v
BindAI API
   |
   v
Configured Agent
   |
   v
Agent Streaming Output
   |
   v
Streaming HTTP Response
   |
   v
Client

The API uses FastAPI’s StreamingResponse to return output from the agent’s stream_chat() method. The endpoint provides an HTTP streaming boundary around BindAI agent execution. It does not define a structured event protocol for the stream.

Endpoint

The streaming endpoint is:

POST /api/v1/agents/{agent_name}/stream

The {agent_name} path parameter identifies the configured agent that should handle the request. For example:

POST /api/v1/agents/researcher/stream

The agent must be available in the BindAI API application’s configured agent collection. If the requested agent cannot be found, the endpoint returns HTTP 404 Not Found.

Authentication

The streaming endpoint is protected by the BindAI API authentication layer. Clients must provide an API key in the Authorization header:

Authorization: Bearer your-secret-key

The server’s API key is configured through the BINDAI_API_KEY environment variable:

BINDAI_API_KEY=your-secret-key

For example, using cURL:

curl -N \
  -X POST \
  -H "Authorization: Bearer your-secret-key" \
  -H "Content-Type: application/json" \
  -d '{"message":"Explain retrieval augmented generation."}' \
  http://localhost:8000/api/v1/agents/researcher/stream

When the server has a configured API key, a missing or invalid client key results in HTTP 401 Unauthorized. If BINDAI_API_KEY is not configured on the server, the authentication dependency raises a server-side configuration error instead. See the Authentication page for details.

Request Body

The streaming endpoint accepts a JSON request containing a message:

{
  "message": "Explain retrieval augmented generation."
}

The message field is passed to the configured agent’s stream_chat() method. Use a string value for the message. Refer to the generated API documentation for the exact request schema exposed by the version you are running.

Streaming Response

The endpoint returns a text-based HTTP streaming response. Conceptually:

Agent starts
    |
    v
Output chunk
    |
    v
Client receives data
    |
    v
More output
    |
    v
Client receives more data
    |
    v
Agent completes
    |
    v
Response ends

The route obtains output from agent.stream_chat(request.message) and yields the output as a stream. Values that are not strings are converted to strings before being yielded. The amount of data received in each chunk depends on the underlying iterator and HTTP transport. Clients must not assume a fixed number of chunks or a particular chunk size.

Streaming with cURL

Use cURL’s -N option to disable its usual output buffering:

curl -N \
  -X POST \
  -H "Authorization: Bearer your-secret-key" \
  -H "Content-Type: application/json" \
  -d '{"message":"Write a short explanation of AI agents."}' \
  http://localhost:8000/api/v1/agents/researcher/stream

Replace researcher with an agent name configured in your application.

Streaming with Python

A Python HTTP client can consume the response incrementally using the requests library:

import requests

url = "http://localhost:8000/api/v1/agents/researcher/stream"

response = requests.post(
    url,
    headers={
        "Authorization": "Bearer your-secret-key",
        "Content-Type": "application/json",
    },
    json={
        "message": "Explain AI agents in simple terms.",
    },
    stream=True,
    timeout=(10, 120),
)

response.raise_for_status()

try:
    for chunk in response.iter_content(
        chunk_size=None,
        decode_unicode=True,
    ):
        if chunk:
            print(chunk, end="", flush=True)
finally:
    response.close()

The example uses a connection timeout of 10 seconds and a read timeout of 120 seconds. Adjust these values to suit your workload. The read timeout is not necessarily a total limit on the duration of the entire stream. The client processes received data incrementally. Depending on the HTTP transport and buffering, one yielded chunk is not guaranteed to correspond to one model token or one complete sentence.

Streaming with JavaScript

A JavaScript application can consume the HTTP response using the Fetch API and a readable stream:

const response = await fetch(
  "http://localhost:8000/api/v1/agents/researcher/stream",
  {
    method: "POST",
    headers: {
      Authorization: "Bearer your-secret-key",
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      message: "Explain AI agents in simple terms.",
    }),
  },
);

if (!response.ok) {
  throw new Error(`Request failed: ${response.status}`);
}

if (!response.body) {
  throw new Error("The response does not provide a readable stream.");
}

const reader = response.body.getReader();
const decoder = new TextDecoder();

try {
  while (true) {
    const { value, done } = await reader.read();

    if (done) {
      break;
    }

    const text = decoder.decode(value, { stream: true });
    console.log(text);
  }

  const remaining = decoder.decode();

  if (remaining) {
    console.log(remaining);
  }
} finally {
  reader.releaseLock();
}

A browser or server-side JavaScript application can append each decoded chunk to its output interface. The example reads bytes from the HTTP stream and decodes them as text; chunk boundaries should not be treated as semantic boundaries.

Streaming vs Normal Execution

BindAI provides separate routes for normal and streaming agent execution. Normal execution:

POST /api/v1/agents/{agent_name}/run

Streaming execution:

POST /api/v1/agents/{agent_name}/stream

The conceptual difference is: Normal execution

Client
   |
   v
Agent
   |
   v
Complete Result
   |
   v
Client

Streaming execution

Client
   |
   v
Agent
   |
   +---- Output chunk
   +---- Output chunk
   +---- Output chunk
   |
   v
Response ends

Normal execution is convenient when the client needs the complete result before continuing. Streaming is useful when the client wants to display output incrementally.

When to Use Streaming

Streaming is useful for interactive applications, including:

Chat interfaces.
Agent consoles.
Interactive assistants.
Long generated responses.
Content-generation interfaces.
Developer tools.

A user interface can display output while the agent is still producing it, rather than waiting for the entire response. Whether output is produced incrementally in practice depends on the configured agent, its underlying execution, and the surrounding infrastructure.

Streaming and Model Providers

The REST API provides the HTTP streaming boundary around agent execution. The configured agent and model provider determine how the underlying execution produces output. Conceptually:

REST API
   |
   v
Agent
   |
   v
Configured Model Provider
   |
   v
Underlying Model Execution

Different providers and agent configurations may produce output at different rates. Clients should treat the response as an incremental text stream rather than relying on provider-specific timing. The REST route itself does not guarantee that every underlying model provider produces tokens incrementally.

Chunk Handling

Clients should treat received data as parts of a larger text response. A chunk may not represent:

A complete sentence.
A complete word.
A complete semantic unit.
A complete application-level event.

For example, the client might receive text in pieces such as:

Chunk 1: "Retrieval"
Chunk 2: " augmented"
Chunk 3: " generation"
Chunk 4: " combines"

These are illustrative chunk boundaries, not a guarantee of the actual output pattern. Clients that need the complete generated text should accumulate the chunks. Avoid building application logic that assumes each chunk is independently meaningful.

Buffering

Streaming can be affected by buffering at different layers:

Agent Execution
      |
      v
BindAI Application
      |
      v
ASGI Server
      |
      v
Reverse Proxy
      |
      v
Client

A reverse proxy or hosting platform may buffer response data even when the application returns a streaming response. When deploying streaming applications, test the complete request path to verify that output reaches clients with the expected latency.

Reverse Proxies

If a reverse proxy is placed in front of BindAI:

Client
   |
   v
Reverse Proxy
   |
   v
BindAI API

the proxy must be configured appropriately for streaming traffic. Otherwise, it may collect response data before forwarding it to the client. The exact configuration depends on the reverse proxy and hosting platform. BindAI does not control infrastructure-level response buffering.

Connection Lifetime

A streaming request keeps its HTTP response open while output is being produced. Conceptually:

Request
   |
   v
Agent starts
   |
   |------ stream active ------|
   |                           |
   v                           v
First output             Final output
                               |
                               v
                         Response ends

Clients should keep the connection open until the stream finishes or an error occurs. Infrastructure should use suitable request and connection timeouts for streaming workloads. A timeout that is appropriate for a short JSON response may be too restrictive for a longer-running stream.

Client Disconnects

A client may disconnect before the response completes. For example:

Client
   |
   | Request
   v
BindAI API
   |
   v
Agent
   |
   +---- Output
   |
   X Client disconnects

Applications should not assume that the client remains connected for the entire execution. The streaming route does not itself provide a durable background job mechanism or a separate run-status endpoint for a disconnected request. The behavior of the underlying agent execution after a disconnect should not be assumed to be equivalent to cancelling a background job. For work that must be submitted and inspected independently of the original HTTP request, consider the background-run API and its process-local execution limitations.

Streaming vs Background Runs

Streaming and background execution solve different problems. Streaming

Client
   |
   v
Agent
   |
   v
Live HTTP output

Background execution

Client
   |
   v
Submit Run
   |
   v
Run ID
   |
   v
Background Execution
   |
   v
Query Run State

Use streaming when the client wants to consume output through an active HTTP response. Use background runs when the client wants to submit work and query the run separately. The v0.1 background-run implementation is process-local and uses in-memory state and history by default. It is not a distributed durable queue. See the Background Runs page for details.

Streaming and Authentication Errors

Authentication is checked before the protected streaming endpoint can be used. When the server has a configured API key, a missing or invalid key results in HTTP 401 Unauthorized rather than a successful stream. If the server’s BINDAI_API_KEY is not configured, the authentication dependency raises a server-side configuration error. A request that passes authentication can still fail because the agent is missing or its execution encounters an error.

Streaming and Agent Errors

A validly authenticated request can still fail during execution. Potential failure sources include:

Agent configuration.
Model provider errors.
Tool errors.
Connection failures.
External-service failures.
Application errors.

Clients should handle unsuccessful HTTP responses and connection failures. A stream that ends unexpectedly should not automatically be interpreted as proof that the agent completed successfully. The current route exposes yielded output as text. It does not define a separate structured error-event protocol within the response stream.

Streaming and Tools

An agent may use tools during execution, depending on its configuration. Conceptually:

Client
   |
   v
Streaming Agent
   |
   +---- Model execution
   |
   +---- Tool execution
              |
              v
        External Service

The streaming endpoint provides the HTTP output stream for agent execution. It should not be treated as a structured event stream containing every internal model, tool, workflow, or connection event. For execution observability, use the runtime event facilities available in the relevant BindAI execution path.

Streaming and Observability

Client-facing streaming output and runtime observability are separate concerns.

HTTP Streaming
      |
      v
Client-facing agent output

Runtime Events
      |
      v
Execution inspection and observability

BindAI Runtime includes an event system and an in-memory EventRecorder for recording events emitted through an execution context’s event bus. Do not assume that every runtime event is included in the HTTP response or that the streaming endpoint exposes all event data. Event availability depends on the execution path and event instrumentation.

Content Type

The current v0.1 streaming route returns a text-based StreamingResponse with the media type text/plain. Clients should consume the response as streamed text. The current implementation is not a Server-Sent Events (text/event-stream) endpoint. It does not define SSE-specific framing, event names, or event IDs. Applications should not assume that each received chunk is a JSON object or a structured event.

Streaming Protocol Scope

The current v0.1 implementation provides a simple HTTP streaming interface. It does not define a complete structured streaming protocol for:

Token metadata.
Tool-call events.
Tool results.
Structured model events.
Usage events.
Workflow events.
Connection events.
Separate in-stream error events.

Applications that require this information should use appropriate runtime instrumentation or implement an application-specific event layer. A richer structured streaming protocol would need to be documented separately from the current plain-text response.

API Documentation

FastAPI exposes API documentation for the running BindAI API application. When running locally, the documentation is commonly available at:

http://localhost:8000/docs

The OpenAPI schema is also available from the running application. The generated documentation reflects the API schema exposed by the installed version. Consult it when checking the exact request model and available routes.

Testing Streaming

The API package test suite can be run with:

uv run pytest packages/bindai-api/tests -q

Relevant streaming tests should cover:

Endpoint availability.
Authentication.
Agent lookup.
Request validation.
Streaming response behavior.
Returned agent output.

The command runs the API package tests; it does not guarantee that every listed behavior has a dedicated test. Provider-specific streaming behavior should be validated separately where required.

Local Development

Start the API locally with:

uv run uvicorn bindai_api.app:app --host 0.0.0.0 --port 8000

Then send a POST request to:

http://localhost:8000/api/v1/agents/{agent_name}/stream

Include the configured API key:

Authorization: Bearer your-secret-key
Content-Type: application/json

And a JSON request body:

{
  "message": "Hello from BindAI."
}

Replace {agent_name} with an agent configured in your application.

Production Considerations

When deploying streaming endpoints publicly, consider:

HTTPS.
Reverse-proxy buffering.
Request and connection timeouts.
Client disconnects.
API authentication.
Rate limiting.
Provider latency.
Concurrent streams.
Resource usage.
Monitoring.

Streaming requests can keep HTTP connections open longer than ordinary request/response operations. Configure infrastructure limits for the expected workload and test streaming through the actual production network path.

Current v0.1 Scope

BindAI v0.1 provides:

Agent streaming through the REST API.
Bearer API-key authentication.
HTTP streaming responses with a text/plain media type.
Incremental output from the agent’s streaming iterator.
Compatibility with HTTP clients that support streamed responses.

The current implementation is intentionally lightweight. It does not define a standardized structured streaming event protocol, and it does not turn the streaming endpoint into a durable background execution system.

Future Streaming Capabilities

Future BindAI releases could expand streaming with capabilities such as:

Structured streaming events.
Token metadata.
Tool-call and tool-result events.
Usage information.
Execution identifiers.
Explicit stream cancellation.
Structured error events.
Server-Sent Events support.
WebSocket support.
Structured workflow streaming.

These are potential extensions, not capabilities promised by the current v0.1 endpoint.

Summary

BindAI v0.1 exposes agent streaming through:

POST /api/v1/agents/{agent_name}/stream

Requests require a bearer API key and a JSON body containing a message string. The endpoint calls the configured agent’s stream_chat() method and returns the yielded output as a text-based HTTP streaming response with media type text/plain. Streaming is useful for interactive applications, but it should not be confused with background execution or runtime observability. The current architecture separates:

Streaming
    |
    v
Client-facing agent output

Background Runs
    |
    v
Separately submitted execution and run-state lookup

Runtime Events
    |
    v
Execution observability

This lightweight streaming API provides a foundation for interactive BindAI applications while leaving structured streaming protocols and richer event delivery outside the current v0.1 implementation.

​Streaming

​Streaming Architecture

​Endpoint

​Authentication

​Request Body

​Streaming Response

​Streaming with cURL

​Streaming with Python

​Streaming with JavaScript

​Streaming vs Normal Execution

​When to Use Streaming

​Streaming and Model Providers

​Chunk Handling

​Buffering

​Reverse Proxies

​Connection Lifetime

​Client Disconnects

​Streaming vs Background Runs

​Streaming and Authentication Errors

​Streaming and Agent Errors

​Streaming and Tools

​Streaming and Observability

​Content Type

​Streaming Protocol Scope

​API Documentation

​Testing Streaming

​Local Development

​Production Considerations

​Current v0.1 Scope

​Future Streaming Capabilities

​Summary